Calibration of Polytomous Item Families Using Bayesian Hierarchical Modeling

For complex educational assessments, there is an increasing use of item families, which are groups of related items. However, calibration or scoring for such an assessment requires fitting models that take into account the dependence structure inherent among the items that belong to the same item family. Glas and van der Linden (2001) suggest a Bayesian hierarchical model to analyze data involving item families with multiple-choice items. This paper extends the model to take into account item families with polytomous items and designs a Markov chain Monte Carlo (MCMC) algorithm for the Bayesian estimation of the model parameters. The hierarchical model, which accounts for the dependence structure inherent among the items, implicitly defines the family response function for the score categories. This paper suggests a way to combine the family response functions over the score categories to obtain a family score function, which is a quick graphical summary of the expected score of an individual with a certain ability on an item randomly generated from an item family. This paper also suggests a method for the Bayesian estimation of the family response function and family score function. This work is a significant step towards building a tool to analyze data involving item families and may be very useful practically, for example, in automatic item generation systems that create tests involving item families.